Method, apparatus and electronic device for processing panoramic image

ABSTRACT

A method, an apparatus and an electronic device for processing a panoramic image are provided. The method includes converting a first planar projection image of the panoramic image into a spherical image; determining a position of a region of interest in the spherical image; and cutting and expanding the spherical image based on the position of the region of interest in the spherical image, and obtaining a second planar projection image of the panoramic image. A distance between the region of interest in the second planar projection image and a center point of the second planar projection image is shorter than a distance between the region of interest in the first planar projection image and a center point of the first planar projection image.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. § 119 to ChineseApplication No. 201710121375.7 filed on Mar. 2, 2017, the entirecontents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to the field of panoramic imageprocessing, and specifically, a method, an apparatus and an electronicdevice for processing a panoramic image.

2. Description of the Related Art

Video information is important for information representation in dailylife. Particularly, with the development of network technology, videotechnology has been applied to daily life, such as video conference,videophone, monitoring and the like.

An ordinary camera cannot capture a multi-directional image and a visualrange of a video is limited, accordingly a panoramic camera is usuallyused for capturing a panoramic image.

A panoramic image can display all image information on an image,accordingly the limitation of visual range can be overcome.Conventionally, when processing a panoramic image, the sphericalpanoramic image is converted into an equidistant cylindrical projection(also called equirectangular projection) image whose aspect ratio is2:1, then image detection is performed for a region of interest (ROI) inthe equidistant cylindrical projection image to obtain imageinformation. However, if the region of interest is close to or locatedin a north pole or a south pole of the panoramic image, large distortionof the image will occur in the region of interest after converting thespherical panoramic image into the equidistant cylindrical projectionimage, accordingly accuracy of the subsequent image detectiondeteriorates.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, a method for processinga panoramic image includes converting a first planar projection image ofthe panoramic image into a spherical image; determining a position of aregion of interest in the spherical image; and cutting and expanding,based on the position of the region of interest in the spherical image,the spherical image, and obtaining a second planar projection image ofthe panoramic image. A distance between the region of interest in thesecond planar projection image and a center point of the second planarprojection image is shorter than a distance between the region ofinterest in the first planar projection image and a center point of thefirst planar projection image.

Furthermore, the first planar projection image is a first equidistantcylindrical projection image. Converting the first planar projectionimage of the panoramic image into the spherical image includes mapping alonger side at the top of the first equidistant cylindrical projectionimage and a longer side at the bottom of the first equidistantcylindrical projection image to a north pole of the spherical image anda south pole of the spherical image, respectively; and merging twoshorter sides of the first equidistant cylindrical projection image at0° longitude of the spherical image.

Furthermore, determining the position of the region of interest in thespherical image includes detecting the position of the region ofinterest in the spherical image by a method of color detection, motiondetection or artificial feature detection.

Furthermore, cutting and expanding the spherical image based on theposition of the region of interest in the spherical image includesdetermining longitude α of a center point of the region of interest inthe spherical image;

determining an intersection A between a meridian of α+180° longitude ofthe spherical image and an equator of the spherical image; cutting thespherical image along a cutting line on the equator, a length of thecutting line being equal to half of a length of the equator, and acenter point of the cutting line being the intersection A; and expandingthe cut spherical image in a first direction and a second direction,respectively, the first direction being a direction from theintersection A to a south pole of the spherical image, and the seconddirection being a direction from the intersection A to a north pole ofthe spherical image.

Furthermore, obtaining the second planar projection image of thepanoramic image includes obtaining, based on pixel values of integerpixels in the first planar projection image, pixel values ofcorresponding points in the second planar projection image, andobtaining, by interpolation or extrapolation based on the pixel valuesof the corresponding points, pixel values of integer pixels adjacent tothe corresponding points in the second planar projection image; ordetermining corresponding points in the first planar projection imagecorresponding to integer pixels in the second planar projection image,obtaining, by interpolation or extrapolation based on pixel values ofinteger pixels adjacent to the corresponding points in the first planarprojection image, pixel values of the corresponding points, andobtaining, based on the pixel values of the corresponding points, pixelvalues of the integer pixels in the second planar projection image.

Furthermore, the method further includes performing image detection forthe region of interest in the second planar projection image, afterobtaining the second planar projection image of the panoramic image.

According to another aspect of the present invention, an apparatus forprocessing a panoramic image includes a first conversion moduleconfigured to convert a first planar projection image of the panoramicimage into a spherical image; a detection module configured to determinea position of a region of interest in the spherical image; and a secondconversion module configured to cut and expand, based on the position ofthe region of interest in the spherical image, the spherical image, andobtain a second planar projection image of the panoramic image. Adistance between the region of interest in the second planar projectionimage and a center point of the second planar projection image isshorter than a distance between the region of interest in the firstplanar projection image and a center point of the first planarprojection image.

Furthermore, the first planar projection image is a first equidistantcylindrical projection image. The first conversion module maps a longerside at the top of the first equidistant cylindrical projection imageand a longer side at the bottom of the first equidistant cylindricalprojection image to a north pole of the spherical image and a south poleof the spherical image, respectively; and merges two shorter sides ofthe first equidistant cylindrical projection image at 0° longitude ofthe spherical image.

Furthermore, the detection module detects the position of the region ofinterest in the spherical image by a method of color detection, motiondetection or artificial feature detection.

Furthermore, the second conversion module includes a first processingunit configured to determine longitude α of a center point of the regionof interest in the spherical image; a second processing unit configuredto determine an intersection A between a meridian of α+180° longitude ofthe spherical image and an equator of the spherical image; a cuttingunit configured to cut the spherical image along a cutting line on theequator, a length of the cutting line being equal to half of a length ofthe equator, and a center point of the cutting line being theintersection A; and an expanding unit configured to expand the cutspherical image in a first direction and a second direction,respectively, the first direction being a direction from theintersection A to a south pole of the spherical image, and the seconddirection being a direction from the intersection A to a north pole ofthe spherical image.

Furthermore, second conversion module includes a pixel value calculatingunit configured to obtain, based on pixel values of integer pixels inthe first planar projection image, pixel values of corresponding pointsin the second planar projection image, and obtain, by interpolation orextrapolation based on the pixel values of the corresponding points,pixel values of integer pixels adjacent to the corresponding points inthe second planar projection image; or determine corresponding points inthe first planar projection image corresponding to integer pixels in thesecond planar projection image, obtain, by interpolation orextrapolation based on pixel values of integer pixels adjacent to thecorresponding points in the first planar projection image, pixel valuesof the corresponding points, and obtain, based on the pixel values ofthe corresponding points, pixel values of the integer pixels in thesecond planar projection image.

Furthermore, the apparatus further includes an image detection moduleconfigured to perform image detection for the region of interest in thesecond planar projection image, after obtaining the second planarprojection image of the panoramic image.

According to another aspect of the present invention, an electronicdevice for processing a panoramic image includes a memory storingcomputer-readable instructions; and one or more processors configured toexecute the computer-readable instructions such that the one or moreprocessors are configured to convert a first planar projection image ofthe panoramic image into a spherical image, determine a position of aregion of interest in the spherical image, and cut and expand, based onthe position of the region of interest in the spherical image, thespherical image, and obtain a second planar projection image of thepanoramic image. A distance between the region of interest in the secondplanar projection image and a center point of the second planarprojection image is shorter than a distance between the region ofinterest in the first planar projection image and a center point of thefirst planar projection image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart illustrating a panoramic image processing methodaccording to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a method of cutting and expanding aspherical image based on a position of a region of interest in thespherical image according to the embodiment of the present invention;

FIG. 3 is a flowchart illustrating a panoramic image processing methodaccording to another embodiment of the present invention;

FIG. 4 is a block diagram illustrating a configuration of a panoramicimage processing apparatus according to an embodiment of the presentinvention;

FIG. 5 is a block diagram illustrating a configuration of a secondconversion module according to an embodiment of the present invention;

FIG. 6 is a block diagram illustrating a configuration of a secondconversion module according to another embodiment of the presentinvention;

FIG. 7 is a block diagram illustrating a configuration of a panoramicimage processing apparatus according to another embodiment of thepresent invention;

FIG. 8 is a schematic diagram illustrating a configuration of anelectronic device for processing a panoramic image according to anembodiment of the present invention;

FIG. 9 is a flowchart illustrating a panoramic image processing methodaccording to a specific embodiment of the present invention;

FIG. 10 is a schematic diagram illustrating a spherical image accordingto the embodiment of the present invention;

FIG. 11 is a schematic diagram illustrating conversion from thespherical image to a second equidistant cylindrical projection imageaccording to the embodiment of the present invention;

FIG. 12 is a schematic diagram illustrating a point (x, y, z) in thespherical image according to the embodiment of the present invention;

FIG. 13 is one of three first equidistant cylindrical projection imagesobtained according to the embodiment of the present invention;

FIG. 14 is a second equidistant cylindrical projection image convertedfrom the first equidistant cylindrical projection image illustrated inFIG. 13 according to the embodiment of the present invention;

FIG. 15 is one of three first equidistant cylindrical projection imagesobtained according to the embodiment of the present invention;

FIG. 16 is a second equidistant cylindrical projection image convertedfrom the first equidistant cylindrical projection image illustrated inFIG. 15 according to the embodiment of the present invention;

FIG. 17 is one of three first equidistant cylindrical projection imagesobtained according to the embodiment of the present invention; and

FIG. 18 is a second equidistant cylindrical projection image convertedfrom the first equidistant cylindrical projection image illustrated inFIG. 17 according to the embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

In the following, specific embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings, so asto facilitate the understanding of technical problems to be solved bythe present invention, technical solutions of the present invention, andadvantages of the present invention.

In view of the problem of the conventional technology, embodiments ofthe present invention have an object to provide a method, an apparatus,and an electronic device for processing a panoramic image that can solveimage distortion occurring in the vicinity of a north pole and a southpole of a panoramic image when converting the panoramic image into aplanar projection image and can improve accuracy of image detection.

First Embodiment

The present embodiment provides a panoramic image processing method. Asillustrated in FIG. 1, the panoramic image processing method includessteps 101 to 103.

Step 101: convert a first planar projection image of the panoramic imageinto a spherical image.

Step 102: determine a position of a region of interest in the sphericalimage.

Step 103: cut and expand the spherical image based on the position ofthe region of interest in the spherical image, and obtain a secondplanar projection image of the panoramic image. Here, a distance betweenthe region of interest in the second planar projection image and acenter point of the second planar projection image is shorter than adistance between the region of interest in the first planar projectionimage and a center point of the first planar projection image.

In the present embodiment, the first planar projection image of thepanoramic image is converted into the spherical image, the sphericalimage is cut and expanded based on the position of the region ofinterest in the spherical image, and the second planar projection imageof the panoramic image is obtained. The distance between the region ofinterest in the second planar projection image and the center point ofthe second planar projection image is shorter than the distance betweenthe region of interest in the first planar projection image and thecenter point of the first planar projection image. Thus, in theconverted second planar projection image, the region of interest iscloser to the center point of the second planar projection image, imagedistortion occurring in the region of interest can be reduced.Accordingly, the accuracy of the image detection can be improved whenperforming image detection for the region of interest in the secondplanar projection image.

The first planar projection image may be an equidistant cylindricalprojection image whose aspect ratio is 2:1, and may also be another typeof projection image. When the first planar projection image is anequidistant cylindrical projection image whose aspect ratio is 2:1, theconverted second planar projection image is an equidistant cylindricalprojection image whose aspect ratio is 1:2.

As an example, when the first planar projection image is a firstequidistant cylindrical projection image, converting the first planarprojection image of the panoramic image into the spherical imageincludes (a) mapping a longer side at the top of the first equidistantcylindrical projection image and a longer side at the bottom of thefirst equidistant cylindrical projection image to a north pole of thespherical image and a south pole of the spherical image, respectively,and (b) merging two shorter sides of the first equidistant cylindricalprojection image at 0° longitude of the spherical image.

When detecting the position of the region of interest in the sphericalimage, the detection may be performed using a conventional ROI (regionof interest) detection method. For example, the position of the regionof interest in the spherical image may be detected by a method of colordetection, motion detection or artificial feature detection.

As an example, as illustrated in FIG. 2, cutting and expanding thespherical image based on the position of the region of interest in thespherical image includes steps 1031 to 1034.

Step 1031: determine longitude α of a center point of the region ofinterest in the spherical image.

Step 1032: determine an intersection A between a meridian of α+180°longitude of the spherical image and an equator of the spherical image.

Step 1033: cut the spherical image along a cutting line on the equator.A length of the cutting line is equal to half of a length of theequator, and a center point of the cutting line is the intersection A.

Step 1034: expand the cut spherical image in a first direction and asecond direction, respectively. The first direction is a direction fromthe intersection A to a south pole of the spherical image, and thesecond direction is a direction from the intersection A to a north poleof the spherical image.

Furthermore, when converting the first planar projection image of thepanoramic image into the second planar projection image of the panoramicimage, it is necessary to determine a pixel value of each of integerpixels in the second planar projection image. Specifically, determiningpixel values of integer pixels in the second planar projection imageincludes (a) obtaining, based on pixel values of integer pixels in thefirst planar projection image, pixel values of corresponding points inthe second planar projection image, and obtaining, by interpolation orextrapolation based on the pixel values of the corresponding points,pixel values of integer pixels adjacent to the corresponding points inthe second planar projection image; or (b) determining correspondingpoints in the first planar projection image corresponding to integerpixels in the second planar projection image, obtaining, byinterpolation or extrapolation based on pixel values of integer pixelsadjacent to the corresponding points in the first planar projectionimage, pixel values of the corresponding points, and obtaining, based onthe pixel values of the corresponding points, pixel values of theinteger pixels in the second planar projection image.

Furthermore, as illustrated in FIG. 3, after step 103, the panoramicimage processing method further includes step 104.

Step 104: perform image detection for the region of interest in thesecond planar projection image. In this way, image distortion occurringin the region of interest in the second planar projection image can bereduced. Accordingly, the accuracy of the image detection can beimproved when performing image detection for the region of interest inthe second planar projection image.

In the embodiment of the present invention, the first planar projectionimage of the panoramic image is converted into the spherical image, thespherical image is cut and expanded based on the position of the regionof interest in the spherical image, and the second planar projectionimage of the panoramic image is obtained. The distance between theregion of interest in the second planar projection image and the centerpoint of the second planar projection image is shorter than the distancebetween the region of interest in the first planar projection image andthe center point of the first planar projection image. Thus, in theconverted second planar projection image, the region of interest iscloser to the center point of the second planar projection image, imagedistortion occurring in the region of interest can be reduced.Accordingly, the accuracy of the image detection can be improved whenperforming image detection for the region of interest in the secondplanar projection image.

Second Embodiment

The present embodiment provides a panoramic image processing apparatus20. As illustrated in FIG. 4, the panoramic image processing apparatus20 includes a first conversion module 21, a detection module 22 and asecond conversion module 23.

The first conversion module 21 converts a first planar projection imageof the panoramic image into a spherical image.

The detection module 22 determines a position of a region of interest inthe spherical image.

The second conversion module 23 cuts and expands the spherical imagebased on the position of the region of interest in the spherical image,and obtains a second planar projection image of the panoramic image.Here, a distance between the region of interest in the second planarprojection image and a center point of the second planar projectionimage is shorter than a distance between the region of interest in thefirst planar projection image and a center point of the first planarprojection image.

In the present embodiment, the first planar projection image of thepanoramic image is converted into the spherical image, the sphericalimage is cut and expanded based on the position of the region ofinterest in the spherical image, and the second planar projection imageof the panoramic image is obtained. The distance between the region ofinterest in the second planar projection image and the center point ofthe second planar projection image is shorter than the distance betweenthe region of interest in the first planar projection image and thecenter point of the first planar projection image. Thus, in theconverted second planar projection image, the region of interest iscloser to the center point of the second planar projection image, imagedistortion occurring in the region of interest can be reduced.Accordingly, the accuracy of the image detection can be improved whenperforming image detection for the region of interest in the secondplanar projection image.

The first planar projection image may be an equidistant cylindricalprojection image whose aspect ratio is 2:1, and may also be another typeof projection image. When the first planar projection image is anequidistant cylindrical projection image whose aspect ratio is 2:1, theconverted second planar projection image is an equidistant cylindricalprojection image whose aspect ratio is 1:2.

As an example, when the first planar projection image is a firstequidistant cylindrical projection image, the first conversion module 21maps a longer side at the top of the first equidistant cylindricalprojection image and a longer side at the bottom of the firstequidistant cylindrical projection image to a north pole of thespherical image and a south pole of the spherical image, respectively,and merges two shorter sides of the first equidistant cylindricalprojection image at 0° longitude of the spherical image.

When detecting the position of the region of interest in the sphericalimage, the detection may be performed using a conventional ROI (regionof interest) detection method. For example, the detection module 22detects the position of the region of interest in the spherical image bya method of color detection, motion detection or artificial featuredetection.

As an example, as illustrated in FIG. 5, the second conversion module 23includes a first processing unit 231, a second processing unit 232, acutting unit 233 and an expanding unit 234.

The first processing unit 231 determines longitude α of a center pointof the region of interest in the spherical image.

The second processing unit 232 determines an intersection A between ameridian of α+180° longitude of the spherical image and an equator ofthe spherical image.

The cutting unit 233 cuts the spherical image along a cutting line onthe equator. A length of the cutting line is equal to half of a lengthof the equator, and a center point of the cutting line is theintersection A.

The expanding unit 234 expands the cut spherical image in a firstdirection and a second direction, respectively. The first direction is adirection from the intersection A to a south pole of the sphericalimage, and the second direction is a direction from the intersection Ato a north pole of the spherical image.

Furthermore, when converting the first planar projection image of thepanoramic image into the second planar projection image of the panoramicimage, it is necessary to determine a pixel value of each of integerpixels in the second planar projection image. Specifically, asillustrated in FIG. 6, the second conversion module 23 further includesa pixel value calculating unit 235.

The pixel value calculating unit 235 (a) obtains, based on pixel valuesof integer pixels in the first planar projection image, pixel values ofcorresponding points in the second planar projection image, and obtains,by interpolation or extrapolation based on the pixel values of thecorresponding points, pixel values of integer pixels adjacent to thecorresponding points in the second planar projection image, or (b)determines corresponding points in the first planar projection imagecorresponding to integer pixels in the second planar projection image,obtains, by interpolation or extrapolation based on pixel values ofinteger pixels adjacent to the corresponding points in the first planarprojection image, pixel values of the corresponding points, and obtains,based on the pixel values of the corresponding points, pixel values ofthe integer pixels in the second planar projection image.

Furthermore, as illustrated in FIG. 7, the panoramic image processingapparatus 20 further includes an image detection module 24.

The image detection module 24 performs image detection for the region ofinterest in the second planar projection image, after obtaining thesecond planar projection image of the panoramic image. In this way,image distortion occurring in the region of interest in the secondplanar projection image can be reduced. Accordingly, the accuracy of theimage detection can be improved when performing image detection for theregion of interest in the second planar projection image.

Third Embodiment

The present embodiment provides an electronic device 30 for processing apanoramic image. As illustrated in FIG. 8, the electronic device 30includes a processor 32, and a memory 34 storing computer-readableinstructions.

When the computer-readable instructions are executed by the processor32, the processor converts a first planar projection image of thepanoramic image into a spherical image, determines a position of aregion of interest in the spherical image, cuts and expands thespherical image based on the position of the region of interest in thespherical image, and obtains a second planar projection image of thepanoramic image. A distance between the region of interest in the secondplanar projection image and a center point of the second planarprojection image is shorter than a distance between the region ofinterest in the first planar projection image and a center point of thefirst planar projection image.

As illustrated in FIG. 8, the electronic device 30 further includes anetwork interface 31, an input device 33, a hard disk drive (HDD) 35,and a display device 36.

Each of ports and each of devices may be connected to each other via abus architecture. The processor 32 such as one or more centralprocessing units (CPUs), and the memory 34 such as one or more memoryunits may be connected via various circuits. Other circuits such as anexternal device, a regulator and a power management circuit may also beconnected via the bus architecture. Note that these devices arecommunicably connected via the bus architecture. The bus architectureincludes a power supply bus, a control bus and a status signal busbesides a data bus. The detailed description of the bus architecture isomitted here.

The network interface 31 may be connected to a network (such as theInternet, a LAN or the like), obtain related data such as a first planarprojection image, and store the related data in the hard disk drive 35.

The input device 33 may receive various commands input by a user, andtransmit the commands to the processor 32 to be executed. The inputdevice 33 may include a keyboard, a click apparatus (such as a mouse ora track ball), a touch board, a touch panel or the like.

The display device 36 may display a result obtained by executing thecommands.

The memory 34 stores programs and data required for running an operatingsystem, and data such as intermediate results in calculation processesof the processor 32.

Note that the memory 34 of the embodiments of the present invention maybe a volatile memory or a nonvolatile memory, or may include both avolatile memory and a nonvolatile memory. The nonvolatile memory may bea read-only memory (ROM), a programmable read-only memory (PROM), anerasable programmable read-only memory (EPROM), an electrically erasableprogrammable read-only memory (EEPROM) or a flash memory. The volatilememory may be a random access memory (RAM), which used as an externalhigh-speed buffer. The memory 34 of the apparatus or the methoddescribed herein includes and is not limited to any other suitablememory.

In some embodiments, the memory 34 stores executable modules or datastructure, their subsets, or their superset, i.e., an operating system(OS) 341 and an application program 342.

The operating system 341 includes various system programs for realizingvarious essential tasks and processing tasks based on hardware, such asa frame layer, a core library layer, a drive layer and the like. Theapplication program 342 includes various application programs forrealizing various application tasks, such as a browser and the like. Aprogram for realizing the method according to the embodiments of thepresent invention may be included in the application program 342.

When the processor 32 invokes and executes the application program anddata stored in the memory 34, specifically the program or instructionsstored in the application program 342, the processor 32 may convert afirst planar projection image of the panoramic image into a sphericalimage, determine a position of a region of interest in the sphericalimage, cut and expand the spherical image based on the position of theregion of interest in the spherical image, and obtain a second planarprojection image of the panoramic image. A distance between the regionof interest in the second planar projection image and a center point ofthe second planar projection image is shorter than a distance betweenthe region of interest in the first planar projection image and a centerpoint of the first planar projection image.

The method according to the above embodiments of the present inventionmay be applied to the processor 32 or may be realized by the processor32. The processor 32 may be an integrated circuit chip capable ofprocessing signals. Each step of the above method may be realized byinstructions in a form of integrated logic circuit of hardware in theprocessor 32 or a form of software. The processor 32 may be ageneral-purpose processor, a digital signal processor (DSP), anapplication specific integrated circuit (ASIC), field programmable gatearray signals (FPGA) or other programmable logic device (PLD), adiscrete gate or transistor logic, discrete hardware components capableof realizing or executing the methods, the steps and the logic blocks ofthe embodiments of the present invention. The general-purpose processormay be a micro-processor, and alternatively, the processor may be anycommon processors. The steps of the method according to the embodimentsof the present invention may be realized by a hardware decodingprocessor, or combination of hardware modules and software modules in adecoding processor. The software modules may be located in aconventional storage medium such as a random access memory (RAM), aflash memory, a read-only memory (ROM), a erasable programmableread-only memory (EPROM), an electrically erasable programmableread-only memory (EEPROM), a register or the like. The storage medium islocated in the memory 34, and the processor 32 reads information in thememory 34 and realizes the steps of the above methods in combinationwith hardware.

Note that the embodiments described herein may be realized by hardware,software, firmware, intermediate code, microcode or any combinationthereof. For hardware implementation, the processor may be realized inone or more application specific integrated circuits (ASIC), digitalsignal processors (DSP), programmable logic devices (PLD), fieldprogrammable gate array signals (FPGA), general-purpose processors,controllers, micro-controllers, micro-processors, or other electroniccomponents or their combinations for realizing functions of the presentinvention.

For software implementation, the embodiments of the present inventionmay be realized by executing functional modules (such as processes,functions or the like). Software codes may be stored in a memory andexecuted by a processor. The memory may be implemented inside or outsidethe processor.

Specifically, the processor 32 maps a longer side at the top of thefirst equidistant cylindrical projection image and a longer side at thebottom of the first equidistant cylindrical projection image to a northpole of the spherical image and a south pole of the spherical image,respectively, and merges two shorter sides of the first equidistantcylindrical projection image at 0° longitude of the spherical image.

Specifically, the processor 32 detects the position of the region ofinterest in the spherical image by a method of color detection, motiondetection or artificial feature detection.

Specifically, the processor 32 determines longitude α of a center pointof the region of interest in the spherical image, determines anintersection A between a meridian of α+180° longitude of the sphericalimage and an equator of the spherical image, cuts the spherical imagealong a cutting line on the equator, a length of the cutting line beingequal to half of a length of the equator, and a center point of thecutting line being the intersection A, and expands the cut sphericalimage in a first direction and a second direction, respectively, thefirst direction being a direction from the intersection A to a southpole of the spherical image, and the second direction being a directionfrom the intersection A to a north pole of the spherical image.

Specifically, the processor 32 obtains pixel values of correspondingpoints in the second planar projection image based on pixel values ofinteger pixels in the first planar projection image, and obtains pixelvalues of integer pixels adjacent to the corresponding points in thesecond planar projection image by interpolation or extrapolation basedon the pixel values of the corresponding points. Alternatively, theprocessor 32 determines corresponding points in the first planarprojection image corresponding to integer pixels in the second planarprojection image, obtains pixel values of the corresponding points byinterpolation or extrapolation based on pixel values of integer pixelsadjacent to the corresponding points in the first planar projectionimage, and obtains pixel values of the integer pixels in the secondplanar projection image based on the pixel values of the correspondingpoints.

Specifically, the processor 32 performs image detection for the regionof interest in the second planar projection image, after obtaining thesecond planar projection image of the panoramic image.

In the present embodiment, the first planar projection image of thepanoramic image is converted into the spherical image, the sphericalimage is cut and expanded based on the position of the region ofinterest in the spherical image, and the second planar projection imageof the panoramic image is obtained. The distance between the region ofinterest in the second planar projection image and the center point ofthe second planar projection image is shorter than the distance betweenthe region of interest in the first planar projection image and thecenter point of the first planar projection image. Thus, in theconverted second planar projection image, the region of interest iscloser to the center point of the second planar projection image, imagedistortion occurring in the region of interest can be reduced.Accordingly, the accuracy of the image detection can be improved whenperforming image detection for the region of interest in the secondplanar projection image.

Fourth Embodiment

As illustrated in FIG. 9, the panoramic image processing methodaccording to the present embodiment includes steps 401 to 404.

In step 401, a first equidistant cylindrical projection image of apanoramic image is converted into a spherical image.

Before step 401, the panoramic image has been converted, by anothersoftware or algorithm, into an equidistant cylindrical projection imagewhose aspect ratio is 2:1, i.e., the first equidistant cylindricalprojection image. The first equidistant cylindrical projection image isconverted into the spherical image by mapping a longer side at the topof the first equidistant cylindrical projection image and a longer sideat the bottom of the first equidistant cylindrical projection image to anorth pole of the spherical image and a south pole of the sphericalimage, respectively, and merging two shorter sides of the firstequidistant cylindrical projection image at 0° longitude of thespherical image.

In the present embodiment, assuming that a width of the firstequidistant cylindrical projection image is 2n units, a height of thefirst equidistant cylindrical projection image is 2n units and acoordinate system is established based on an upper left vertex of thefirst equidistant cylindrical projection image serving as an origin, acoordinate of each pixel in the first equidistant cylindrical projectionimage can be represented by (θ, Φ), where 2π≥θ≥0, π≥Φ≥0. Afterconverting the first equidistant cylindrical projection image into thespherical image, θ is longitude of each pixel, and Φ is latitude of eachpixel.

The first equidistant cylindrical projection image is mapped onto asurface of the spherical image by exporting the longitude and latitudeof each pixel. The conversion is performed by the following equation.

$\theta = {\frac{{{index}\mspace{14mu} {of}\mspace{14mu} Y} + 0.5}{\# \mspace{14mu} {of}\mspace{14mu} {cols}}*\pi}$$\varphi = {\frac{{{index}\mspace{14mu} {of}\mspace{14mu} X} + 0.5}{\# \mspace{14mu} {of}\mspace{14mu} {rows}}*2\pi}$

Where (X, Y) is a coordinate of a pixel in the first equidistantcylindrical projection image, an index of the pixels starts from (0, 0),the obtained spherical image is shown in FIG. 10, where longitude of aregion of interest a, i.e., a detection object is represented by ψ.

In step 402, a position of the region of interest in the spherical imageis determined.

In this step, approximate longitude of the region of interest may beobtained. The method of determining the region of interest in thespherical image includes and is not limited to color detection, motiondetection of a region of interest in a video frame, or artificialfeature detection combined with another additional sensor (a depthsensor or a binocular camera). The color detection means determining theposition of the region of interest in the spherical image using color ofthe region of interest different from color of background environment.For example, in a case of face recognition, it may be determined that aregion whose color is skin color is the region of interest. In a case ofmotion object detection, it may be determined that a region where motionoccurs is the region of interest.

After the region of interest is determined, longitude of a center pointof the region of interest may be obtained by the following method.

An average value of longitude of all pixels or a part of pixels of theregion of interest is calculated as the longitude of the center point ofthe region of interest. Alternatively, a probability distributionfunction of effective pixels of the region of interest is obtained, andan average value with a predetermined confidence range is obtained asthe longitude of the center point of the region of interest.

If the region of interest is located in the vicinity of a north pole anda south pole of the first equidistant cylindrical projection image,accuracy limitation for longitude estimation can be reduced. That is tosay, in a converted second equidistant cylindrical projection image, thedegree of image distortion of a region of interest located at highlatitude in the spherical image is relatively low.

In step 403, the spherical image is cut and expanded based on theposition of the region of interest in the spherical image, and a secondequidistant cylindrical projection image is obtained.

If the longitude of the region of interest is ψ, the spherical image iscut along an equator between ψ+π/2 meridian and ψ+3π/2 meridian, the cutspherical image is expanded in a direction towards the north pole and adirection towards the south pole, respectively, and the secondequidistant cylindrical projection image is obtained. As illustrated inFIG. 11, the spherical image is converted into the second equidistantcylindrical projection image whose aspect ratio is 1:2. In FIG. 11, “1”is the first equidistant cylindrical projection image, “2” is thespherical image, “3” is the second equidistant cylindrical projectionimage, and “a” is the region of interest.

As illustrated in FIG. 12, a calculation formula of a three-dimensionalposition coordinate (x, y, z) of each point in the spherical image is asfollows, where (θ, Φ) is a coordinate of each pixel in the firstequidistant cylindrical projection image.

x=sin θ cos ϕ

y=sin θ sin ϕ

z=cos θ

A three-dimensional position coordinate (x′, y′, z′) in the sphericalimage that corresponds to (x, y, z) may be represented as follows, whereψ is the longitude of the region of interest.

x′=−x cos ψ+y sin ψ

y′=x sin ψ−y cos ψ

z=z′

A coordinate (θ′, Φ′) of a point in the converted second equidistantcylindrical projection image that corresponds to point (θ, Φ) in thefirst equidistant cylindrical projection image may be represented asfollows.

$\varphi^{\prime} = {{{arc}\; \tan \; 2\left( {z^{\prime},x^{\prime}} \right)\mspace{14mu} {if}\mspace{14mu} \theta} \in \left\lbrack {0,\frac{\pi}{2}} \right\rbrack}$$\varphi^{\prime} = {{{2\; \pi} + {{arc}\; \tan \; 2\left( {z^{\prime},x^{\prime}} \right)\mspace{14mu} {if}\mspace{14mu} \theta}} \in \left\lbrack {\frac{\pi}{2},\pi} \right\rbrack}$θ^(′) = arc tan  2(x^(′)cos  φ^(′) + z^(′)sin  φ^(′), y^(′))

The corresponding points (θ′, Φ′) are unevenly distributed in the secondequidistant cylindrical projection image whose aspect ratio is 1:2.

In an interpolation or extrapolation step of pixels, pixel values ofinteger pixels in the second equidistant cylindrical projection imagemay be obtained by the following methods.

(1) obtain pixel values of all corresponding points in the second planarprojection image based on pixel values of integer pixels in the firstplanar projection image, and obtain pixel values of integer pixels inthe second planar projection image by interpolation or extrapolationbased on the pixel values of all corresponding points. The interpolationalgorithm includes and is not limited to double linear interpolation,nearest neighbors interpolation, cubic interpolation and the like.

(2) determine corresponding points in the first planar projection imagecorresponding to integer pixels in the second planar projection image,obtain pixel values of the corresponding points by interpolation orextrapolation based on pixel values of integer pixels adjacent to thecorresponding points in the first planar projection image, and obtainpixel values of the integer pixels in the second planar projection imagebased on the pixel values of the corresponding points.

In step 404, image detection is performed for the region of interest inthe second equidistant cylindrical projection image.

Finally, object detection is performed for the region of interest in theconverted second equidistant cylindrical projection image whose aspectratio is 1:2. Since image distortion occurring in the region of interestin the second equidistant cylindrical projection image is relativelylow, the accuracy of the image detection can be improved.

The present invention can solve the problem of image distortionoccurring in the, vicinity of a north pole and a south pole of apanoramic image, restore image distortion regions close to the northpole or the south pole, and improve detectable rate or accuracy of imagedetection. The embodiments of the present invention can be applied tovarious scenes using a panoramic camera.

As illustrated in FIGS. 13, 15 and 17, in three first equidistantcylindrical projection images obtained according to the presentembodiment, since the images obtained by a panoramic cameraequidistantly stretch in the vicinity of a north pole or a south pole,severe distortion occurs at the top and bottom of the first equidistantcylindrical projection images, and it is difficult to detect faces by aface detection algorithm. On the other hand, as illustrated in FIGS. 14,16 and 18, in the second equidistant cylindrical projection imagesconverted using the technology according to the present invention, theimage distortion can be greatly reduced, and the accuracy of the imagedetection can be greatly improved.

The present invention is not limited to the specifically disclosedembodiments, and various modifications, combinations and replacementsmay be made without departing from the scope of the present invention.

What is claimed is:
 1. A method for processing a panoramic image, themethod comprising: converting a first planar projection image of thepanoramic image into a spherical image; determining a position of aregion of interest in the spherical image; and cutting and expanding,based on the position of the region of interest in the spherical image,the spherical image, and obtaining a second planar projection image ofthe panoramic image, wherein a distance between the region of interestin the second planar projection image and a center point of the secondplanar projection image is shorter than a distance between the region ofinterest in the first planar projection image and a center point of thefirst planar projection image.
 2. The method for processing a panoramicimage according to claim 1, wherein the first planar projection image isa first equidistant cylindrical projection image, and wherein convertingthe first planar projection image of the panoramic image into thespherical image includes mapping a longer side at the top of the firstequidistant cylindrical projection image and a longer side at the bottomof the first equidistant cylindrical projection image to a north pole ofthe spherical image and a south pole of the spherical image,respectively; and merging two shorter sides of the first equidistantcylindrical projection image at 0° longitude of the spherical image. 3.The method for processing a panoramic image according to claim 1,wherein determining the position of the region of interest in thespherical image includes detecting the position of the region ofinterest in the spherical image by a method of color detection, motiondetection or artificial feature detection.
 4. The method for processinga panoramic image according to claim 1, wherein cutting and expandingthe spherical image based on the position of the region of interest inthe spherical image includes determining longitude α of a center pointof the region of interest in the spherical image; determining anintersection A between a meridian of α+180° longitude of the sphericalimage and an equator of the spherical image; cutting the spherical imagealong a cutting line on the equator, a length of the cutting line beingequal to half of a length of the equator, and a center point of thecutting line being the intersection A; and expanding the cut sphericalimage in a first direction and a second direction, respectively, thefirst direction being a direction from the intersection A to a southpole of the spherical image, and the second direction being a directionfrom the intersection A to a north pole of the spherical image.
 5. Themethod for processing a panoramic image according to claim 1, whereinobtaining the second planar projection image of the panoramic imageincludes obtaining, based on pixel values of integer pixels in the firstplanar projection image, pixel values of corresponding points in thesecond planar projection image, and obtaining, by interpolation orextrapolation based on the pixel values of the corresponding points,pixel values of integer pixels adjacent to the corresponding points inthe second planar projection image; or determining corresponding pointsin the first planar projection image corresponding to integer pixels inthe second planar projection image, obtaining, by interpolation orextrapolation based on pixel values of integer pixels adjacent to thecorresponding points in the first planar projection image, pixel valuesof the corresponding points, and obtaining, based on the pixel values ofthe corresponding points, pixel values of the integer pixels in thesecond planar projection image.
 6. The method for processing a panoramicimage according to claim 1, the method further comprising: performingimage detection for the region of interest in the second planarprojection image, after obtaining the second planar projection image ofthe panoramic image.
 7. An apparatus for processing a panoramic image,the apparatus comprising: a first conversion module configured toconvert a first planar projection image of the panoramic image into aspherical image; a detection module configured to determine a positionof a region of interest in the spherical image; and a second conversionmodule configured to cut and expand, based on the position of the regionof interest in the spherical image, the spherical image, and obtain asecond planar projection image of the panoramic image, wherein adistance between the region of interest in the second planar projectionimage and a center point of the second planar projection image isshorter than a distance between the region of interest in the firstplanar projection image and a center point of the first planarprojection image.
 8. The apparatus for processing a panoramic imageaccording to claim 7, wherein the first planar projection image is afirst equidistant cylindrical projection image, and wherein the firstconversion module maps a longer side at the top of the first equidistantcylindrical projection image and a longer side at the bottom of thefirst equidistant cylindrical projection image to a north pole of thespherical image and a south pole of the spherical image, respectively;and merges two shorter sides of the first equidistant cylindricalprojection image at 0° longitude of the spherical image.
 9. Theapparatus for processing a panoramic image according to claim 7, whereinthe detection module detects the position of the region of interest inthe spherical image by a method of color detection, motion detection orartificial feature detection.
 10. The apparatus for processing apanoramic image according to claim 7, wherein the second conversionmodule includes a first processing unit configured to determinelongitude α of a center point of the region of interest in the sphericalimage; a second processing unit configured to determine an intersectionA between a meridian of α+180° longitude of the spherical image and anequator of the spherical image; a cutting unit configured to cut thespherical image along a cutting line on the equator, a length of thecutting line being equal to half of a length of the equator, and acenter point of the cutting line being the intersection A; and anexpanding unit configured to expand the cut spherical image in a firstdirection and a second direction, respectively, the first directionbeing a direction from the intersection A to a south pole of thespherical image, and the second direction being a direction from theintersection A to a north pole of the spherical image.
 11. The apparatusfor processing a panoramic image according to claim 7, wherein thesecond conversion module includes a pixel value calculating unitconfigured to obtain, based on pixel values of integer pixels in thefirst planar projection image, pixel values of corresponding points inthe second planar projection image, and obtain, by interpolation orextrapolation based on the pixel values of the corresponding points,pixel values of integer pixels adjacent to the corresponding points inthe second planar projection image; or determine corresponding points inthe first planar projection image corresponding to integer pixels in thesecond planar projection image, obtain, by interpolation orextrapolation based on pixel values of integer pixels adjacent to thecorresponding points in the first planar projection image, pixel valuesof the corresponding points, and obtain, based on the pixel values ofthe corresponding points, pixel values of the integer pixels in thesecond planar projection image.
 12. The apparatus for processing apanoramic image according to claim 7, the apparatus further comprising:an image detection module configured to perform image detection for theregion of interest in the second planar projection image, afterobtaining the second planar projection image of the panoramic image. 13.An electronic device for processing a panoramic image, the electronicdevice comprising: a memory storing computer-readable instructions; andone or more processors configured to execute the computer-readableinstructions such that the one or more processors are configured toconvert a first planar projection image of the panoramic image into aspherical image; determine a position of a region of interest in thespherical image; and cut and expand, based on the position of the regionof interest in the spherical image, the spherical image, and obtain asecond planar projection image of the panoramic image, wherein adistance between the region of interest in the second planar projectionimage and a center point of the second planar projection image isshorter than a distance between the region of interest in the firstplanar projection image and a center point of the first planarprojection image.